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Abstract 

The two main contributions of this paper are a proof of concept of the recent novel idea in the area of long-time average cost 
control, and a new method of overcoming the well-known difficulty of non-convexity of simultaneous optimization of a control 
law and an additional tunable function. A recently-proposed method of obtaining rigorous bounds of long-time average cost is 
first outlined for the uncontrolled system with polynomials of system state on the right-hand side. In this method the polynomial 
constraints are relaxed to be sum-of-squares and formulated as semi-definite programs. It was proposed to use the upper bound of 
long-time average cost as the objective function instead of the time-average cost itself in controller design. In the present paper 
this suggestion is implemented for a particular system and is shown to give good results. Designing the optimal controller by this 
method requires optimising simultaneously both the control law and a tunable function similar to the Lyapunov function. The 
new approach proposed and implemented in this paper for overcoming the inherent non-convexity of this optimisation is based 
on a formal assumption that the amplitude of control is small. By expanding the tunable function and the bound in the small 
parameter, the long-time average cost is reduced by minimizing the respective bound in each term of the series. The derivation 
of all the polynomial coefficients in controller is given in terms of the solvability conditions of state-dependent linear and bilinear 
inequalities. The resultant sum-of-squares problems are solved in sequence, thus avoiding the non-convexity in optimization. The 
proposed approach is implemented for a simple model of oscillatory vortex shedding behind a cylinder. 

Keywords: Sum of squares; Long-time average; Polynomial systems; Small feedback; Non-convexity 


1. Introduction 

Although global stabilization of dynamical systems is of importance in system theory and engineering Qi], it is 
sometimes difficult or impossible to synthesize a global stabilizing controller for certain linear and nonlinear systems 
13]. The reasons could be the poor controllability of system, e.g., systems that have uncontrollable linearizations 
14 ] and systems that have fewer degrees of control freedom than the degrees of freedom to be controlled llH, the 
input/output constraints in practice, e.g., an unstable linear time-invariant system cannot be globally stabilized in the 
presence of input saturations (01, time delay (00, and/or the involved large disturbances JT^l, etc. Moreover, in 
many applications the full stabilization, while possible, carries high penalty due to the cost of the control, thus is also 
not desirable. 
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Instead, minimizing a long-time average of the cost functional might be more realistic. For instance, long-time 
average cost analysis and control is often considered in irrigation, flood control, navigation, water supply, hydro 

[F 


electric power, computer communication networks, and other applications yj 


. In addition, systems that include 
13tl, a summary of long-time-average 


stochastic factors are often controlled in the sense of long-time av erag e. In _ ^ 

cost problems for continuous-time Markov processes is given. In jldll . the long-time-average control of a class of 
problems that arise in the modeling of semi-active suspension systems was considered, where the cost includes a term 


14|] are highly 


based on the local time process diffusion. Notice that the controller design methods proposed in Il3 
dependent on the stochastic property of dynamical systems. 

In certain cases, as, for example, turbulent flows of fluid, calculating the time averages is a big challenge even in 
the uncontrolled case. As a result, developing the control aimed at reducing the time-averaged cost for turbulent flows, 
for example by using the receding horizon technique, leads to controllers too complicated for practical implementation 
Q.To overcome this complexity, it was proposed Q to use an upper bound for the long-time average cost instead 
of the long-time average cost itself in cases when such an upper bound is easier to calculate. The idea is based on the 
hope that the control reducing an upper bound for a quantity will also reduce the quantity itself. Meanwhile, iQ uses 
the sum of squares (SOS) decomposition of polynomials and semidefinite programming (SDP) and allows a trade-off 
between the quality of bound and the complexity of its calculation. 

The SOS methods apply to systems defined by a polynomial vector held. Such systems may describe a wide 
variety of dynamics iQ or approximate a system defined by an analytical vector held fj]. A polynomial system can 
therefore yield a reliable model of a dynamical system globally or in larger regions than the linear approximation 
in the state-space Q. Recent results on SOS decomposition have transformed the verification of non-negativity of 
polynomials into SDP, hence providing promising algorithmic procedures for stability analysis of polynomial systems. 
However, using SOS techniques for optimal control, as for example in lElSI EH] , is subject to a generic difficulty: 
while the problem of optimizing the candidate Lyapunov function certifying the stability for a closed-loop system for a 
given controller and the problem of optimizing the controller for a given candidate Lyapunov function are reducible to 
an SDP and thus, are tractable, the problem of simultaneously optimizing both the control and the Lyapuniov function 
is non-convex. Iterative procedures were proposed for overcoming this difficulty j 


M\2M- 


While optimization of an upper bound with control proposed in does not involve a Lyapunov function, it 
does involve a similar tunable function, and it shares the same difficulty of non-convexity. In the present work we 
propose a polynomial type state feedback controller design scheme for the long-time average upper-bound control, 
where the controller takes the structure of an asymptotic series in a small-amplitude perturbation parameter. By fully 
utilizing the smallness of the perturbation parameter, the resultant SOS optimization problems are solved in sequence, 
thus avoiding the non-convexity in optimization. We apply it to an illustrative example and demonstrate that it does 
allow to reduce the long-time average cost even without fully stabilizing the system. Notice the significant conceptual 
difference between our approach and the studies of control by small perturbations, often referred to as tiny feedback, 
see for example lIFll . 

The paper is organized as follows. Section |2] presents some preliminary introduction on SOS and its application 
in bound estimation of long-time average cost for uncontrolled systems. Section [3] gives the problem formulation. 
Bound optimization of the long-time average cost for controlled polynomial systems is considered in Section]?] An 
illustrative example of a cylinder wake flow is addressed in Section]5] Section]2concludes the work. 


2. Background 

In this section SOS of polynomials and a recently-proposed method of obtaining rigorous bounds of long-time 
average cost via SOS for uncontrolled polynomial systems are introduced. 

2.7. SOS of polynomials 

SOS techniques have been frequently used in the stability analysis and controller design for all kinds of systems, 
e.g., constrained ordinary differential equation systems 12], hybrid systems lE2] . time-delay systems lEhl] . and partial 
differential equation systems SI E2.E2] . These techniques help to overcome the common drawback of approaches 
based on Lyapunov functions: before llOll . there were no coherent and tractable computational methods for construct¬ 
ing Lyapunov functions. 
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A multivariate polynomial /(x) is a SOS, if there exist polynomials /i (x), • • ■, /m(x) such that 


fix) = ^/;"(x). 

i'=i 


If fix) is a SOS then fix) > 0, Vx. In the general multivariate case, however, fix) > 0 Vx does not necessarily imply 
that fix) is SOS. While being stricter, the condition that fix) is SOS is much more computationally tractable than 
non-negativity S. At the same time, practical experience indicates that in many cases replacing non-negativity with 
the SOS property leads to satisfactory results. 

In the present paper we will utilize the existence of efficient numerical methods and software 32] for solving 
the optimization problems of the following type: minimize the linear objective function 


T 

W C 


( 1 ) 


where w is the vector of weighting coefficients for the linear objective function, and c is a vector formed from the 
(unknown) coefficients of the polynomials pfx) for i - 1,2, • ■ •, A and SOS pfx) for i - iN + \), - ■ ■ ,N, such that 

ao,/x) -H Piix)aijix) = 0, ; = 1,2, • ■ •, /, (2) 

aojix) + Zti fKxjai./x) are SOS, y = (/ H- 1), • ■ •, 7. (3) 

In (l2]i and Q, the aifx) are given scalar constant coefficient polynomials. 

The lemma below that provides a sufficient condition to test inclusions of sets defined by polynomials is frequently 
used for feedback controller design in Section |4] It is a particular case of the Positivstellensatz Theorem ll^ and is 
a generalized <S-procedure Isdl] . 

Lemma 1. Consider two sets ofx. 

Si = {xeR”|/z(x) = 0,/i(x)>0,---,/,(x)>0}, 

.S 2 = {xeR”|/o(x)>0!, 

where fix), i — 0, - • ■ ,r and hix) are scalar polynomial functions. The set containment <Si C .S 2 holds if there exist a 
polynomial function mix) and SOS polynomial functions S fx), i — 1, • • •, r such that 

r 

/o(x) -Y^S iix)fix) + mix)hix) is SOS. 

1=1 


2.2. Bound estimation of long-time average cost for uncontrolled systems 

For the convenience of the reader we outline here the method of obtaining bounds for long-time averages proposed 
in ES and make some remarks on it. Consider a system 

X = f(x), (4) 

where x = dxjdt and f(x) is a vector of multivariate polynomials of the components of the state vector x e R". The 
long-time average of a function of the state <t)(x) is defined as 

1 

ffi^lim- 0(x(f))r/f, 

T^oo T Jo 

where x(f) is the solution of ©. 

Define a polynomial function of the system state, y(x), of degree dy, and containing unknown decision variables 
as its coefficients. The time derivative of V along the trajectories of system (IH is 

y(x) = X ■ v^y(x) = f(x) ■ v^y(x). 
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Consider the following quantity: 


Hix) = V{x) + d)(x) = f(x) ■ V^VCx) + <I)(x). 


The following result is from d: 


Lemma 2. For the system assume that the state x is bounded in D c R". Then, H{x) < C,'ix e D implies d) < C. 

Hence, an upper bound of O can be obtained by minimizing C over V under the constraint H{x) < C, which can 
be formulated as a SOS optimization problem in the form: 


min C 

V 


(5) 


s.t. - (f(x) ■ V^V{x) + (l)(x) - C) is SOS, 


( 6 ) 


which is a special case of ([TJ. A better bound might be obtained by removing the requirement for V(x) to be a 
polynomial and replacing (|6ll with the requirement of non-negativeness. However, the resulting problem would be too 
difficult, since the classical algebraic-geometry problem of verifying positive-definiteness of a general multi-variate 
polynomial is NP-hard flSl- 

Notice that while V is similar to a Lyapunov function in a stability analysis, it is not required to be positive-definite. 
Notice also that a lower bound of any long-time average cost of the system (|4li can be analyzed in a similar way. 

Remark 1. For many systems the boundedness of system state immediately follows from energy consideration. In 
general, if the system state is bounded, this can often be proven using the SOS approach. It suffices to check whether 
there exists a large but bounded global attractor, denoted by D\. As an example, let T>\ — {x | 0.5x^x < jS\, where the 
constant f is sufficiently large. Then, the global attraction property of system in T)\ may be expressed as 


x^x = x^f(x) < -(0.5x^x -p). 


(7) 


Introducing a tunable polynomial S (x) satisfying 5 (x) > 0 Vx e R", by Lemma 1, (0 can be relaxed to 


{ -(x^f(x)-5(x)(0.5x^x-/?)) is SOS, 
\ S(x)is SOS. 


( 8 ) 


Minimization of upper bound of long-time average cost for systems that have unbounded global attractor is usually 
meaningless, since the cost itself could be infinitely large. 


3. Problem Formulation 


Consider a polynomial system with single input 


X = f(x) H- g(x)u 


(9) 


where f(x) : R" —> R" and g(x) : R” —> R"^™ are polynomial functions of system state x. The approach of this paper 
can easily be extended to multiple input systems. The control u e R™, which is assumed to be a polynomial vector of 
the system state x with maximum degree d^, is designed to minimize the upper bound of an average cost of the form: 



( 10 ) 


where x is the closed-loop solution of the system 0 with the control u. The continuous function <t) is a given non¬ 
negative polynomial cost in x and u. 
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Similarly to dSl)-®, we consider the following optimization problem: 


minC 

u,V 

s.t. - ((f(x) + g(x)u) ■ VxV + <1)(X, u) - C) is SOS. 


( 11 ) 

( 12 ) 


When it cannot be guaranteed that the closed-loop system state is bounded, SOS constraints dUl must be added to (fT^ 
to make our analysis rigorous. 

Under the framework of SOS optimization, the main problem in solving (fTTT i- lfTSl i is due to the non-convexity of 
(IT 2 I 1 caused by the control input u and the decision function V, both of which are tunable, entering (fTSI) nonlinearly. 
Iterative methods i^l 1^ 13 may help to overcome this issue indirectly in the following way: first fix one subset 
of bilinear decision variables and solve the resulting linear inequalities in the other decision variables; in the next 
step, the other bilinear decision variables are fixed and the procedure is repeated. For the particular long-time average 
cost control problem (fTTT l- lfT^ . the non-convexity will be resolved in the following by considering a type of so-called 
small-feedback controller. In such a new way, iterative updating of decision variables is exempted, and replaced by 
solving a sequence of SOS optimization problems. 


4. Bound optimization of long-time average cost for controlled polynomial systems 

In this section a small-feedback controller is designed to reduce the upper bound of the long-time average cost 
(fTOl) for the controlled polynomial system (|3). It is reasonable to hope that a controller reducing the upper bound for 
the time-averaged cost will also reduce the time-averaged cost itself M- 

4.1. Basic formalism of the controller design 
We will look for a controller in the form 


u(x, e) = ^ e'u,(x), (13) 

(=1 

where e > 0 is a parameter, and u,(x), i - 1, 2, • ■ ■ are polynomial vector functions of system state x. In other words, 
we seek a family of controllers parameterised by e in the form of a Taylor series in e. Notice that the expansion starts 
at the first-order term, so that e - 0 gives the uncontrolled system. To resolve the non-convexity problem of SOS 
optimization, we expand V and C in e: 


V(x,e) = J]e'y,(x), (14) 

(=0 

00 

C(e) = (15) 

i=0 

where V, and C, are the Taylor series coefficients for the tunable function and the bound, respectively, in the /th-order 
term of e. Define 


F(V, u, C) - (f(x) -H g(x)u) ■ VxV + <l>(x, u) - C. 
Substituting (fT3l l. (fT4l i. and (flSl l into (fThl l. we have 


F(V, u, C) = 


/ 00 \ 


f + g^e'u, ■ ^ e'VxV,-H O x,^e'u, - ^ e'C, 


V /=l / /=() 


V /=1 / 


1=0 


/ 00 \ 


d) 


,'I]"' = 2 ' 

V / 1=0 \k=0 

5 


(16) 


Noticing 
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it follows that 

OO 

F(V, u, C) = e‘Fi(Vo, ■ ■ •, V,-, m, ■ • ■, u,-, Q), (17) 

1=0 


where 


= f ■ V,y, + gu, ■ V,y, + l||(x,0)il^(x,0) - c, 


d' 




k=0 


k\ 


de' 


( 18 ) 


In (fTST l. ((9^<l)/5u^)(x, 0) denotes the A:th partial derivative of O with respect to u at u = 0, and (£l'(u^)/5e')(x, 0) denotes 
the ith partial derivative of u^(x, e) = [Mj(x, e), ■ • ■, e)Y with respect to e at e = 0. 

Expression (fTTI i becomes more clear when a specific cost function d) is considered. For instance, let O = <l)o(x) + 
u^u. Then, 


F{V, u, C) = Fq{Vq, Co) + eEi (Vo, Vi , Ui , Ci) + €^F 2 (Vo, Vi, Vz, Ui , Uz, Cz) + 0(€^), 


where 


Fo - f ■ VxVo + <50 - Co, 

Fi = f -Vxyi+gui-Vxyo-Ci, 

Cz = f-Vxyz + gui-Vxyi+guz-Vxyo + u[ui-Cz, 

and O(e^) denotes all the terms with order of e being equal or greater than 3. 

It is clear that F(V, u, C) < 0 holds if < 0, / = 0,1,2, • • simultaneously, and the series (O-dnii converge. 
Notice that F, includes tunable functions Vj,J < i, and Uk,k < i - 1. For any non-negative integers ii,iz satisfying 
ii < iz, the tunable variables in F,, are always a subset of the tunable variables in Fi^. Hence (fTTb - (fT^ can be solved as 
a sequence of convex optimization problems. When the inequality constraints F, < 0 are relaxed to SOS conditions, 
our idea can be summarized as follows. 


The sequential steps to solve i l7il )-( l72l ).' A-I 


(sO) First minimize Co over Fq under the constraint Fo{Vo, Co) < 0, or more conservatively, 

Oo : min Co, s.t. -Fo{Vo,Co) is SOS. 

^0 

Denote the optimal Co by Co,sos and the associated Vo by Vo^sos ■ 

(si) Now,letyo = Vo^sos inFi, and then minimize Ci overFi andui under the constraint Fi(yo,s os, yi,Ui,Ci)<0, 
or under the framework of SOS optimization, 

Oi : minCi, s.t. - Fi(yosos, ^i,Ui, Ci) is SOS. 

Vj.ui 

Using the generalized ,S-procedure given in Femma 1 and the fact that 

- Fo{Vo,sos,Co,sos) '2:0, (19) 

Oi can be revised by incorporating one more tunable function S o(x): 


0 \ : 


S.t. 


niiny|,u,,So Cl, 

-7^1 (Fo,s os,^i,ui,Ci)-i-5 o(x)Fo(yo,s os , Co,s os ) is SOS, 
5o(x) is SOS. 


Denote the optimal Ci by Ci,soj and the associated Vi and Ui by y^sos and Ui,soj, respectively. 

6 
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(s 2 ) Further let Vo = Vo^sos, Vi - Vusos^ and Ui = Ui,sos in F2, and then minimize C2 over V2 and U2 under the 
constraint F'2(yo,so5j ^1,5os> V2, Ui_sos, U2, C2) < 0 . In a more tractable way, consider 

min C2, s.t. 

O2 ■ '^ 2 , U2 

-F2(Vo^sos, yi,sos> Vi, Ui.soj; U2, C2) is SOS. 

Similarly as in (si), noticing (fT9l l and 


-FiiVo^sos, yi,sos> ui,sos> Ci.sos) ^ 0, 

the SDP problem O 2 can be revised by the generalized .S-procedure to the following form: 


02- 


niinv,_u,_s„_s, C 2 , sJ. 

-F2iVo^sos, ^ 1 , 5 os > V’2,u 1,Sos>U2, C2) + So{x)Fi){Vo,SOS,Co,SOs) 
+ S i(x)Fi(yo,sos7 yi,sos>ui,sos> C'i,sos) is SOS, 

5o(x) is SOS, 

S i(x) is SOS. 


Denote the optimal C 2 by €2,5 os and the associated y 2 and U 2 by y 2 ,sos and U 2 ,sos. respectively. 

Notice that S o(x) here might differ from the tunable function S o(x) in (9j. Throughout this paper we will use 
the same notations for the tunable functions like S 0 and S \ in various instances of the .S-procedure, to keep the 
notation simple. 


(s3) The SOS-based controller design procedure is continued for higher-order terms. 


Now, define three series 

CO CO 00 

Csos = ^ usos = ^ y^os = ^ e'y/.sos- (20) 

(=0 r=l /=0 

When all of them converge, the following statement will be true. 

Theorem 1. By applyin^he state-feedback controller u = Usos far the system m, if the trajectories of the closed- 
loop system are boundedii then Cgos an upper bound of the long-time average cost d). 

Proof. Using the algorithm A-I, we obtain 

FiiVo^sos, • • • > y/.sos.Ui.sos. • • • >u/.sos. Qsos) < 0, V i. 

Then, it follows that 

CO 

^ FiiVo^sOSr • •. yi,SOS.Ul,SOS. • • • Q^os) = ■P'(ysos,U505, C^os) ^ 0, 

!=0 

where Cgos?Usos, Vsos are given in (l20l i. By virtue of a same analysis as in proving Lemma 2 (see ifl^ L we can 
conclude that €> < Cjo^. ■ 

Remark 2. After specifying the structure of controller to be of the form ( 1731 ). the non-convexity in solving the optimiza¬ 
tion problem m-m has been avoided by solving the linear SDPs Oq, 0[, O'^, ■■■ in sequence. During the pro cess, 
all the involved decision variables are optimized sequentially, but not iteratively as in other methods h2(X 12211231/ . 

Remark 3. The smallness ofe can be used to relax 0\ , 0'2, ■ ■ - further. For instance, in 0\, in order to prove Fo-t-eFi < 
0, we prove FfYo^sos^ Ti. Ui, Ci) < 0 with the aid of the known constraint 7^o(yo,sos. Co.sos) ^ 0, thus not using 
that e is small. In fact, when e is small, for Fq -¥ eF\ to be negative F\ has to be negative only for those x where 
Fq{x) is small, and not for all x as required in Of Meanwhile, checking the convergence of the series m would be 
challenging or even impractical. These points will be addressed in what follows. 

*In the context of long-time average cost controller design and analysis, it is actually enough to assume the boundedness of the global attractor 
of the system to ensure the existence of Csos ■ 
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4.2. Design of small-feedback controller 

Next, the sequential design method A-I is revised to utilize that e 1. 

The revised sequential steps to solve ( Ii7l )-( I72I ); A-II 

(sO) Same as in A-I, first solve the SOS optimization problem 0(). Denote the optimal Co by Co,sos and the associ¬ 
ated Vo by Vo, 5 os- 

(si) Let Vo = Vo.sos in and then consider the following SDP problem: 


min C 1 , 


s.t. -7^i(Vo,sos>^i>ui>Ci) + Sq(x)Fo(Vo^sos,Cq^sos ) is SOS, 


where 5o is any tunable polynomial function of x of fixed degree. Denote the optimal Ci by Ci^sos and the 
associated Vi and ui by Vi^sos and respectively. Unlike Of here the non-negativity requirement of 5o 

is not imposed. This can be understood as that the non-negativity constraint is imposed only for x such that 

Fo(Vo,sos,Co,sos) - 0 . 

(s2) Further let Vo = Vq.sos, Vi — and Ui = Ui.joj in 7^2, and then consider 


min C 2 , s.t. 

^21^2,ISOM'S" 1 

-7^2(Uo,SOS> V’2,U OJ > U2, C 2 ) -I- S o(x)Fo{Vo^soS, Co,SOS ) 

+ 5i(x)7^i(Vo,sos>^i,sos>ui,sos>C'i_sos) is SOS, 



where S 0 and S 1 are any tunable polynomial functions of fixed degrees. S 0 here does not need to be the same 
as in Of Denote the optimal C 2 by € 2,5 os and the associated V 2 and U 2 by V 2 .SOS and U 2 ,sos^ respectively. 
Similarly as in Of here the non-negativity constraint is in effect imposed only where 7^o(Fo,sos> Co.sos) = 
7^l(i^,SOS J i^l.SOS- Ul, SOS; Cl,5os) = 0. 

(s3) The revised SOS-based controller design procedure is continued for higher-order terms. 


Since the constraints of S being SOS imposed in A-I are removed in A-II, the coefficients Ci^sos obtained in A-II 
can be smaller than the coefficients C,,sos obtained in A-I. This advantage comes at a price: even if all the relevant 
series converge for a particular value of e, the procedure A-II does not guarantee that the value Cgos given in (l20l i 
is an upper bound for the time-averaged cost of the closed-loop system with the controller u^os. We have now to 
consider (l20l) as asymptotic expansions rather than Taylor series. Accordingly, we have to truncate the series and hope 
that the resulting controller will work for (sufficiently) small e[l. It is possible to prove that this is, indeed, the case. 
For illustration, the first-order truncation is considered only. 

Theorem 2. Consider the first-order small-feedback controller for the system m. 


Usos - eul.sos 


( 21 ) 


where e > 0 is sufficiently small. Assume that the trajectories of the closed-loop system are bounded, and that 
C\.sos < 0- Then, Ck^sos = Co.sos + ^ (0; 1) an upper bound of the long-time average cost O. Clearly, 

Ck.sos < Co.sos- 


^It is worthy of noticing that the series truncation here does not mean that our controller design and analysis are conducted in a non-rigorous 
way. The truncated controller would be effective if it leads to a better (lower) bound of the long-time average cost. 
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Proof. Let Vgos - Vo.sos + ^y\,sos■ By substituting V - Vsos,C - Ck.sos , u = u^os in the constraint function 
F{V, u, C) that is defined in (fTbl) . the remaining task is to seek small e > 0 such that 




( 22 ) 


Notice that 

■P’(L5os>usos, = -FoCVo.sos; Q.sos) + e-P’i(Lo,sos. Li,sos.ui,sos> Ci.sos) + e(l - F)C\^sos + e), (23) 

where 


1 / 5<h \ 

w(x,e) = gui ■ Vxyi,sos + ^ I<h(x, eui,sos) - <h(x,0) - e—(x,0)ui,sos I, 

and Fq, F\, being polynomial in x, possess all the continuity properties implied by the proof. Let 2) e R" be the phase 
domain that interests us, where the closed-loop trajectories are all bounded. Then, 

F\^max - niaxT'i(yo,soSjyi,50SjUl,SOS>Ci,sos) < OO, (24) 

xeD 

and w(x, e) is bounded for any x £ D and any finite e (the latter following from the standard mean-value-theorem- 
based formula for the Lagrange remainder). By (|2^ and (l24l) . 

■P'(yjos.usos> Q.joj) ^ Fo(Vo,sos,Co,sos) + ^Fi^,„ax H- e(l - k)Ci^sos + 0(€^). (25) 


Meanwhile, consider the two inequality constraints obtained by solving Oo and (9"; 

( Fo{Vq^sos,Co^sos) ^ 0 , 

I T’i(yo,sos> yi,sos>ui.sos-Ci^sos) < 0 Vx such that Fo(yo,sos, Co.sos) = 0. 


(26) 


Define Dg = {xe D\6 < Fo{Vo,sos,Co,sos) ^ 0} for a given constant b < 0. Clearly, Ds Do as 6 ^ 0. 
Further define 


■P'i,< 5(5) - inaxF’i(yo,joj,yi,soj,ui,soj7Ci,soj)- (27) 

xe®4 

By the second constraint in (l26l l. lim^^o ^ 0. Therefore, by continuity and the fact Ci^sos < 0, for any 

0 < x < 1 there exists a constant < 0 such that 


■P’i(yo,sos>yi,sos>ui,sos.Ci,sos) ^ 7^1,5, < “ 2 ^^ “x)Ci,sos. ^x e 2)^^. 

In consequence, (|2^ . the first constraint in (l26l l. and (1281) render to 

F(Vsos,usos,C,t.sos) ^ Fo(Vo,sos,Co,sos) + ^Fig^ +£(1 -/c)Ci^sos + O(e^) 
< ^(l-^)Ci,sos+O(e^)<0, VxeD,^, 


(28) 


(29) 


for sufficiently small e. 

Next, we prove (l22]i for any x £ 2) \ By the definition of the set we have 

Fo(Vo,sos, Fo,sos) < < 0, Vx £ 2) \ 2)^^. 


(30) 


Then, (l25T l and (l30l l yield 


F(Vsos^Usos,F^.sos) ^ + eFi^max + £(1 - x)Ci,5os + O(e^) < + 0(e) <0, Vx £ 2) \ Ds^, 


(31) 


if e is sufficiently small. 
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(l29l l and OTT i imply that (l22l i holds Vx e £). The proof is complete. ■ 

In practice, once the form of the controller has been specified in (|2TI) . the upper bound C and the corresponding V 
actually can be obtained by solving the following optimization problem directly; 


O,: 


minC, 

V, f 

sJ. - ^(1/; eui,5os. Q is SOS. 


This problem can be further relaxed by incorporating the known constraints (l26l l. In O^, if e is set as one of the tunable 
variables, the SOS optimization problem will become non-convex again, thus causing additional trouble in solving it. 
Alternatively, one can fix e here, and investigate its effect on the upper bound of O by trial and error. We will follow 
this route in Section|5] 


5. Illustrative example 


As an illustrative example we consider a system proposed in ll^ as a model for studying control of oscillatory 
vortex shedding behind a cylinder. The actuation was assumed to be achieved by a volume force applied in a compact 
support region downstream of the cylinder. The Karhunen-Loeve (KL) decomposition ll^ was used and the first two 
KL modes and an additional shift mode were selected. For the Reynolds number equal to 100 the resulting low-order 
Galerkin model of the cylinder flow with control was given as follows 




CTr 

-(x> - ya^ 

-fax 

a\ 


^1 

a2 


a> + yaT, 

CTr -/?a2 

a2 

+ 

82 

^3 


aa\ 


-0-3 



0 


where cr, = 0.05439, crj = 0.05347,0' = 0.02095,/? = 0.02116, y = -0.03504, w = 0.9232, = -0.15402, and 

g 2 = 0.046387. More details on deriving the reduced-order model (l32l i are given in ll37ll . 


The system (l32] | possesses a unique equilibrium when m = 0, which is at the origin. Let <& = 1 /2a^ a + where 
a = [fli 02 The proposed algorithms A-I and A-II were applied to (l32l l. with the system state assumed to be 
available. In experiment, it could be estimated by designing a state observer with some sensed output measurement at 
a typical position 03711 . 


5.7. Performance of algorithm A-I 

The SDP problem Oq is solved first. It corresponds to the uncontrolled sysytem. The minimal upper bound we 
could achieve was Co,sos - 6.59. It was obtained with 


Vq^sos — —96.63o3 + 14.01o^ -t I 4 .OI 02 -(- 14.15o2- 

Increasing the degree of Vo cannot give a better bound because there exists a stable limit cycle in the phase space of 
(I 32 I 1 . on which a^ + a^ = 6.560, and 03 = 2.570. Since O = l/2a^a = 6.584 on the limit cycle, the minimal upper 
bound achieved by SOS optimization is tight in the sense that the difference between Cq.soj €> is less than the 
prescribed precision for C, 0.01. 

Solving the SDP problem 6>i, where Vi andwi are tunable functions, gave Cijoj =0. Solving 6>j, with Vi, mi, 5o 
being tuning functions, gave the same result: Ci^sos - 0. In both cases, increasing the degrees of the tuning functions 
did not reduce the upper bound. The consequent SOS optimization problems, O', with i -2,3 also gave Ci^sos -0,i - 
2,3. Therefore, by (l20l i. 


Csos - Cq^sos + fCrsos + e^C 2 ,sos + O(e^) ^ Cq.sos - 6.59, 

implying that A-I does not generate a control ensuring a better upper bound of O than the bound obtained in the 
uncontrolled case. 


10 















D. Huang et ai. / 00 (2015) j47?l 


11 


5.2. Performance of algorithm A-W 

Without any control, it has been obtained in A-I that Cq.sos = 6.59. 

We first solve O”. Given the vectors of monomials in x without repeated elements OSll . Z,-, i = 1, ■ • •, 3, define 
Vi - P\Zi, u\ - P\Z 2 , and So = P^Zj,, where the parametric vectors P,, i - consist of tuning vector 

variables. The degrees of Vi, mi and 5o are specified by the maximum degrees of monomials in Z,, / = 1, • • ■, 3, and 
denoted by t/y,, du,, and ds„, respectively. Consider two subcases: dy, - du, - t/so - 2 and liy, = du, - = 4. For 

the former case, we have Ci^sos - -354, induced by 


ii\gos,2 — 45.376[i — 28.47u: 2 — 142.76ti2ti3 + 399.49tiiti3. 

For the latter case, we have Ci ,505 = -1965, induced by 

Ml,505,4 = 233.08ai - 54.73a2 - 67.61a2a3 + 218 .560103 + 717.280[ + 13.16oj02 
+ 571 . 67 oi 02 - 277.7302 + 466.6I01O3 - I4I.4I02O3 + 230 . 530[03 
+ 106 . 32 o[O 2 O 3 + 22O.I901O2O3 - I6I.4402O3 +628.4O01O3 - I73.7802O3. 

We then solve with a fixed e. For simplicity we considered u\ sos - mi,505,2 and dy < 10 only. The upper- 
bound results for different e are summarized in Fig. [1] The long-time average cost O, which is obtained by direct 
numerical experiment, and the linear truncated bound Co,sos + fCi.sos are also presented for comparison. From Fig. 
[ 1 ] we can see the following. 

Let ei = 1.267 x 10“^ and 62 — 7.416 x 10'^. The small-feedback controller 

M = £U\.sos ,2 (33) 

reduces O, and the reduction in 4) increases monotonically with e when 0 < e < 62 - In particular, 4) = 0 for ei < e < 62 , 
that is in this range of e the controller fully stabilizes the system. When e > 62 , the controller makes the long-time 
average cost worse than in the uncontrolled case. The effect of e on 4) can be seen more clearly by investigating 
the qualitative properties of the closed-loop system. A simple check gives that when 0 < e < ei, the closed-loop 
system has a unique unstable equilibrium at the origin and a stable limit cycle, thus yielding a non-zero but finite 
4); when ei < e < € 2 , the limit cycle disappears and the unique equilibrium becomes globally stable, thus implying 
the vanishness of 4); when e > €2 but is close to € 2 , besides the equilibrium at the origin, there exist four additional 
non-zero equilibria, and as a result 4> becomes large immediately. For instance, at the bifurcation point e - € 2 , the 
non-zero equilibria of the closed-loop system are (+0.6988, ±2.362,2.377) and (±0.7000, ±2.364,2.382), resulting in 
4) = 171.55. 

Solving Ge,0 < e < 8.7 x 10“^ yields a tight upper bound C^^sos for 4). However, the obtained upper bound 
becomes non-tight when e > 8.7 x 10 The conservativeness of C^.sos can be fully overcome by considering 
additional relaxation constraint (l26l l for 8.7 x 10“^ < e < 4 x 10“^, but only mitigated to certain extend for larger e. 

The two-term expansion Co,SOS + fCl, 5 05 is only a linear approximation of Cjos in (l 20 l l. Thus, as an upper bound 

of 4), it behaves well when e is very small, but it becomes conservative when e is further increased, and meaningless 
as e > —Co.sos ICi^sos — 0.0186, 

In summary, for small e, the proposed small-feedback controller yields a better bound of the long-time average 
cost than in the uncontrolled case. Further, the controller indeed reduces the long-time average cost itself. 

Figs. |33]show more details of the control performance of the proposed controller (l33t with e = 8.7 x 10“^ and 
the initial state a = [-0.3 - 0.3 0.3]^. 

6. Conclusion 

Based on sum-of-squares decomposition of polynomials and semidefinite programming, a numerically tractable 
approach is presented for long-time average cost control of polynomial dynamical systems. The obtained controller 
possesses a structure of small feedback, which is an asymptotic expansion in a small parameter, with all the coeffi¬ 
cients being polynomials of the system state. The derivation of the small-feedback controller is given in terms of the 
solvability conditions of state-dependent linear and bilinear inequalities. The non-convexity in SOS optimization can 
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Figure 1. The long-time average cost O and its upper bounds for different 6. Cq^sos ^Ci^sos ^Ce,sos ->^'^$08 are obtained by solving Oo,Oj',Oe, 
and Of with the relaxation ED, respectively. 
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Figure 2. Control input profile. 



Figure 3. Closed-loop trajectory starting at a = [-0.3 — 0.3 0.3]^. Owing to the small-feedback control, the magnitude of the periodic oscillation 
has been reduced. 
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be resolved by making full use of the smallness of the perturbation parameter while not using any iterative algorithms. 
The efficiency of the control scheme has been tested on a low-order model of cylinder wake flow stabilization problem. 
In the next research phase, we will consider SOS-based long-time average cost control under modelling uncertainties 
and in the presence of noise, as well as direct numerical simulations of small-feedback control for actual fluid flows. 

The proof of concept of the idea of using the upper bound of long-time average cost control as the objective of the 
control design, and the method of overcoming the non-convexity of simultaneous optimization of the control law and 
the tunable function are the two main contributions of the present paper. 
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